Skip to content

Conversation

@lima-limon-inc
Copy link
Collaborator

@lima-limon-inc lima-limon-inc commented Nov 3, 2025

Closes: #121

This PR adds artifact support, both for executables and libraries. With these changes, midenup should first try to download an artifact if it exists, if it doesn't or the download process fails, then it defaults to install the component from source.

Implementation notes:

  • The Artifact struct holds a URI which is used to fetch the artifact, either from a URL (prefixed by https://) or from a file (prefixed by file://). Each artifact only holds a specific triplet (for more information, see LLVM's triplet implementation), which is displayed in the URI itself, for example:
    • https://github.com/0xMiden/miden-debug/releases/download/v0.4.4/miden-debug-aarch64-apple-darwin: : Holds the miden-debugger's binary for aarch64-apple-darwin
    • https://github.com/0xMiden/miden-vm/releases/download/0.0/std.masp: Holds the Package for the std library. (Do note that masp packages are used by the MidenVM and therefore these URIs do not have a platform dependent triplet).
  • The src/external.rs file holds functions that are imported into the install script by the include_str macro. These were added in a separate file to enhance readability.
  • As a final note, now if a component fails to get installed, midenup will simply skip said component if it's in release mode. In debug mode, it panics.

@lima-limon-inc lima-limon-inc self-assigned this Nov 3, 2025
@lima-limon-inc lima-limon-inc added command:midenup An issue or task related to the `midenup` command check:install PRs only: runs workflows that perform additional end-to-end integration testing labels Nov 3, 2025
@lima-limon-inc lima-limon-inc changed the base branch from main to fabrizioorsi/i113-remove-hardcoded-components November 3, 2025 20:59
@lima-limon-inc lima-limon-inc force-pushed the fabrizioorsi/i121-artifact-support branch 4 times, most recently from 77a15c8 to 2fb1bef Compare November 6, 2025 14:47
@lima-limon-inc
Copy link
Collaborator Author

Hi there @bitwalker. Status update: This PR is almost ready for review, I'm doing some final polishing.

Before merging this, I think it would be worthwhile to add an artifact field to at least one component, so that we can check that it works properly. So far, I've been testing this with a simple hello world binary.

Maybe we could try adding the debugger, since that's where the release github action currently resides. Is the next branch ready for merging? Did you have something else in mind?
Do let me know if I can help with anything!

Comment on lines +40 to +72
// bin/ directory which holds binaries.
let bin_dir = toolchain_dir.join("bin");
if !bin_dir.exists() {
std::fs::create_dir_all(&bin_dir).with_context(|| {
format!("failed to create toolchain directory: '{}'", bin_dir.display())
})?;
}

// lib/ directory which holds MASP libraries.
let lib_dir = toolchain_dir.join("lib");
if !lib_dir.exists() {
std::fs::create_dir_all(&lib_dir).with_context(|| {
format!("failed to create toolchain directory: '{}'", lib_dir.display())
})?;
}

// opt/ directory which holds symlinks to binaries in bin/.
// These are used in order to preserve a "midenup" compatible
// interface. This relies on the fact that clap uses argv[0] in order to
// display executable names names. These symlinks have the following format:
// `miden <component name>`
// Then, when `miden` is invoked, it uses these symlinks to execute the
// underlying binary. With this setup, `clap` displays the name as: `miden
// <component name>` instead of just `binary_name` when displaying help
// messages.
Copy link
Collaborator Author

@lima-limon-inc lima-limon-inc Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opt/, bin/ and lib/ did not change in this PR. I simply added these lines and comments to elaborate on their functionality.

@lima-limon-inc lima-limon-inc force-pushed the fabrizioorsi/i121-artifact-support branch 2 times, most recently from f24bcb3 to 0a4ba74 Compare November 6, 2025 17:33
@lima-limon-inc lima-limon-inc marked this pull request as ready for review November 6, 2025 20:47
@lima-limon-inc lima-limon-inc force-pushed the fabrizioorsi/i121-artifact-support branch from d184473 to c82f722 Compare November 6, 2025 20:47
{%- else if dep.path %}, path = "{{ dep.path }}"
{%- endif %} }
{%- endfor %}
curl = "{{ curl_version }}"
Copy link
Collaborator Author

@lima-limon-inc lima-limon-inc Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will use the same version of CURL as the midenup itself. This is obtained in the build.rs file.

"artifacts": [
{
"target": "zkvm-miden-unknown",
"uri": "https://github.com/lima-limon-inc/fabriz.io/releases/download/0.0/std.masp"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These URLS are not intended to be merged and are only here to test this PR. They should be removed before merging.

Base automatically changed from fabrizioorsi/i113-remove-hardcoded-components to main November 10, 2025 12:46
@lima-limon-inc lima-limon-inc force-pushed the fabrizioorsi/i121-artifact-support branch from 30d37dd to d0d2f32 Compare November 13, 2025 17:52
@lima-limon-inc
Copy link
Collaborator Author

Sidenote: I'll give the implementation a rework. I'll replace the triplet to url table with a list of urls instead to simplify things.

@lima-limon-inc lima-limon-inc marked this pull request as draft November 14, 2025 14:50
@lima-limon-inc lima-limon-inc marked this pull request as ready for review November 25, 2025 15:52
@bitwalker
Copy link
Collaborator

@lima-limon-inc I finally got the the release workflow for miden-debug sorted with artifact (with attestation) support working, with the v0.4.4 release.

v0.4.4 can be used with the v0.19 toolchain, upcoming releases will work with v0.20+. In the meantime, hopefully that provides all that is needed to validate a real component works with this feature.

@lima-limon-inc lima-limon-inc force-pushed the fabrizioorsi/i121-artifact-support branch 2 times, most recently from a747d42 to 68d1af9 Compare January 12, 2026 17:58
@lima-limon-inc lima-limon-inc force-pushed the fabrizioorsi/i121-artifact-support branch from 68d1af9 to f6eec13 Compare January 12, 2026 19:06
@lima-limon-inc lima-limon-inc force-pushed the fabrizioorsi/i121-artifact-support branch 2 times, most recently from 081b138 to 501a0f9 Compare January 12, 2026 19:39
Signed-off-by: Tomas Fabrizio Orsi <[email protected]>
Signed-off-by: Tomas Fabrizio Orsi <[email protected]>
Signed-off-by: Tomas Fabrizio Orsi <[email protected]>
Signed-off-by: Tomas Fabrizio Orsi <[email protected]>
Signed-off-by: Tomas Fabrizio Orsi <[email protected]>
Signed-off-by: Tomas Fabrizio Orsi <[email protected]>
@lima-limon-inc lima-limon-inc force-pushed the fabrizioorsi/i121-artifact-support branch from 501a0f9 to 4d295a2 Compare January 12, 2026 19:46
@lima-limon-inc
Copy link
Collaborator Author

@lima-limon-inc I finally got the the release workflow for miden-debug sorted with artifact (with attestation) support working, with the v0.4.4 release.

v0.4.4 can be used with the v0.19 toolchain, upcoming releases will work with v0.20+. In the meantime, hopefully that provides all that is needed to validate a real component works with this feature.

@bitwalker awesome, thanks for the ping!

I have just added the debugger in the 0.19.0 toolchain with its corresponding artifact field in this commit: a85fb79.

It only required doing a small change to the way URI's were parsed, here: 4d295a2. Since the link did not contain the component name (i.e. "debugger"), but rather the repository name (i.e. "miden-debug"). However, this ended up simplifying the implementation all together and actually made the URI's more versatile, since now it can be an arbitrary string as long as the triplet is respected.

I tested on my machine (Apple M3) and the debugger's binary was fetched successfully and I was able to execute it. Here's the output with the corresponding command:

MIDENUP_MANIFEST_URI=file://path/to/local/channel-manifest.json MIDENUP_DEBUG_MODE=true ./midenup install stable

Installing: std.masp
     No artifact found. Proceeding to install from source
     Installed!
Installing: base.masp
     No artifact found. Proceeding to install from source
     Installed!
Installing: vm
     No artifact found. Proceeding to install from source
     Installed!
Installing: client
     No artifact found. Proceeding to install from source
     Installed!
Installing: midenc
     No artifact found. Proceeding to install from source
     Installed!
Installing: cargo-miden
     No artifact found. Proceeding to install from source
     Installed!
Installing: node
     No artifact found. Proceeding to install from source
     Installed!
Installing: debugger
     Fetching artifact
     Installed!

MIDENUP_MANIFEST_URI=file://path/to/local/channel-manifest.json ./miden debugger

WARNING: debugger is installed, but it is not part of the current active toolchain.
error: the following required arguments were not provided:
  <FILE>

(Sidenote: the warning message stems from #123, which got fixed in #124).

@bitwalker
Copy link
Collaborator

@lima-limon-inc Nice work!

The next release of the compiler will have artifacts as well, just like miden-debug, so that should help with install time considerably.

One thing to note, miden-debug is the name of the executable (corresponding to the convention for commands that are surfaced under the miden CLI, i.e. we'd expect to run the debugger using miden debug). I see that the component name for the debugger you chose was debugger, but I think it might be best for component names to stick close to their executable name, when the component is an executable.

This would mean that we'd end up with the following components currently:

  • midenc
  • cargo-miden
  • miden-vm
  • miden-debug
  • miden-client
  • miden-faucet (or whatever the executable ends up being called)

The non-executable component names are a bit trickier, though I think currently there is only the standard library and protocol library that fall in this bucket. For those, I think we'll want to use the convention of naming the component after the package, e.g. core (std was renamed to core in 0.20) and base (the name of this package will likely change too).

Anyway, wanted to align on that since it came up.

@lima-limon-inc
Copy link
Collaborator Author

@bitwalker thanks ^_^.

One thing to note, miden-debug is the name of the executable (corresponding to the convention for commands that are surfaced under the miden CLI, i.e. we'd expect to run the debugger using miden debug). I see that the component name for the debugger you chose was debugger, but I think it might be best for component names to stick close to their executable name, when the component is an executable.

Noted, I've pushed this commit 819ab9f which changes the component's name to debug so that it more closely resembles the underlying binary.

The non-executable component names are a bit trickier, though I think currently there is only the standard library and protocol library that fall in this bucket. For those, I think we'll want to use the convention of naming the component after the package, e.g. core (std was renamed to core in 0.20) and base (the name of this package will likely change too).

Great, thanks for the heads up!

@bitwalker
Copy link
Collaborator

I'll do one last once-over of this today, and try to get it merged ASAP!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

check:install PRs only: runs workflows that perform additional end-to-end integration testing command:midenup An issue or task related to the `midenup` command

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: add artifact support

3 participants